1. Technical Field
The present invention relates generally to communication protocols between a host computer and an input/output (I/O) device. More specifically, the present invention provides a hardware implementation for offloading management of a send queue. In particular, the present invention provides a mechanism by which work requests are turned into work queue entries (WQEs) and are passed from Upper Layer Protocol (e.g. sockets) software to an Internet Protocol (IP) Suite Offload Engine (IPSOE). The present invention also provides a mechanism by which completed WQEs are passed back to the Upper Layer Protocol (ULP) software. The present invention also provides a mechanism for supporting Selective Acknowledgments.
2. Description of Related Art
In an Internet Protocol (IP) Network, the software provides a message passing mechanism that can be used to communicate with input/output devices, general purpose computers (host), and special purpose computers. The message passing mechanism consists of a transport protocol, an upper level protocol, and an application programming interface. The key standard transport protocols used on IP networks today are the Transmission Control Protocol (TCP) and the User Datagram Protocol (UDP). TCP provides a reliable service and UDP provides an unreliable service. In the future the Stream Control Transmission Protocol (SCTP) will also be used to provide a reliable service. Processes executing on devices or computers access the IP network through upper level protocols, such as Sockets, iSCSI, and Direct Access File System (DAFS).
Unfortunately, the TCP/IP software consumes a considerable amount of processor and memory resources. This problem has been covered extensively in the literature (see J. Kay, J. Pasquale, “Profiling and reducing processing overheads in TCP/IP”, IEEE/ACM Transactions on Networking, Vol. 4, No. 6, pp. 817-828, December 1996; and D. D. Clark, V. Jacobson, J. Romkey, H. Salwen, “An analysis of TCP processing overhead”, IEEE Communications Magazine, volume: 27, Issue: 6, Jun. 1989, pp 23-29). In the future the network stack will continue to consume excessive resources for several reasons, including: increased use of networking by applications; use of network security protocols; and the underlying fabric bandwidths are increasing at a higher rate than microprocessor and memory bandwidths. To address this problem, the industry is offloading the network stack processing to a TCP/IP Offload Engine (TOE).
There are two offload approaches being taken in the industry. The first approach uses the existing TCP/IP network stack, without adding any additional protocols. This approach can offload TCP/IP to hardware, but unfortunately does not remove the need for receive side copies. As noted in the papers above, copies are one of the largest contributors to central processing unit (CPU) and memory bandwidth utilization. To remove the need for copies, the industry is pursuing the second approach that consists of adding Framing, Direct Data Placement (DDP), and Remote Direct Memory Access (RDMA) over the TCP and SCTP protocols.
Today the host CPU performs most IP suite processing. TCP/IP Offload Engines (TOE) offer a higher performance interface for communicating to other general purpose computers and I/O devices. Data sends or receives through the TOE require that the CPU either copy data from one memory location to another or register the memory so that the TOE can directly access the memory region. Each of these options requires significant CPU resources with the memory registration option being preferred for large memory transfers, however, as network speeds increase the amount of CPU resources required will increase. A simple mechanism is needed to implement Send Queue in the TOE and perform RDMA, DDP, framing, and TCP/IP processing in the TOE. The mechanism needs to maintain all RDMA, DDP, framing, TCP, IP, and Ethernet state in the TOE. It must also provide the necessary protection to support out of user space Send Queue operations. The present invention, hereinafter referred to as an Internet Protocol Suite Offload Engine (IPSOE), also provides a mechanism for supporting Selective Acknowledgments.